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SUMMARY 

Risk  assessment  has  traditionally  been  accomplished  through  a  variety  of  means. 
Analysis  of  military  tasks,  environments,  and  requirements  has  provided  insight 
into  the  cognitive  components  of  work  and  has  allowed  the  creation  of  models. 
Models,  particularly  sequential  network  models,  have  then  been  used  to  simulate 
the  work  tasks  under  varying  conditions  and  performai;;a  levels  to  make  estimates 
of  the  military  effect  of  degradation  of  task  components.  Observation  of  soldiers  in 
the  field,  coupled  vdth  administration  of  questionnaires,  has  provided  estimates  of 
the  potential  degradation  of  performance  for  a  variety  of  factors,  such  as  pain, 
overheating,  and  nausea  commonly  associated  by  battlefield  weapons  of  mass 
destruction.  Measurement  of  performance  using  standardized  controlled  tests  have 
provided  quantitative  data  to  back  up  these  estimates  and  has  provided  verification 
of  the  risk  estimates.  These  tests,  traditionally  assembled  into  more  generic 
performance  assessment  batteries,  have  progressed  in  recent  years  to  computer- 
based  simulations  of  work-related  tasks. 

This  report  describes  efforts  oondueted  under  the  Risk  Assessment  Tools  contract 
design  and  develop  a  new  tool,  called  the  Synthetic  Task  Authoring  $ystem,  for 
creating  synthetic  tasks  or  part-task  simulators.  It  also  describes  efforts  to 
improve,  automate,  correlate,  and  provide  better  access  to  a  variety  of  tools. 

The  Risk  Assessment  Tools  contract  not  only  continued  the  important  work  done  by 
DNA’s  Intermediate  Dose  Program  but  produced  an  extensible  authoring  tool, 
SYNTAS,  for  test  instruments  that  will  simplify  the  data  gathering  phase  of 
subsequent  work. 

SYNTAS  gives  DN  A 

□  A  practical  way  to  estimate  soldier  performance  decrements  due  to  nudear, 
chemical,  biological,  or  conventional  stresses 

□  An  e’jctensible  platform  for  developing  models,  gaming  scenarios,  training 
systems,  and  assessment  instruments  with  human  performance  included 

□  An  exportable  product  to  other  agendes  and  services  for  similar  uses. 

SYNTAS  has  been  shown  to  be  capable  of  produdng  quite  complex  part-task 
mmulators.  However,  to  date  no  organization  has  us^  SYNTAS  for  actual 
construction  of  research  instruments.  This  is  partially  due  to  the  very  dosed 
development  and  review  community. 

We  recommend  that  efforts  be  made  to  promote  SYNTAS  outside  the  immediate 
development  community  and  encourage  its  practical  use. 
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The  initial  implementation  of  SYNTAS  centers  on  two-dimensional  simulations  on 
a  computer  screen.  This  is  adequate  for  a  large  range  of  tasks  but  stops  short  of 
SYNTAS*  potential. 

Logical  extensions  to  SYNTAS  can  provide: 

□  A  means  of  modeling  crew  and  team  performance  using  multiple 
interconnected  SYNTAS  applications 

□  Increased  fidelity  through  incorporation  of  video,  high-quality  sound,  and 
techniques  from  virtual  reality 

□  Through  a  linkage  to  Micro  SAINT,  authoring  of  SYNTAS  applications  baaed 
on  Micro  SAINT  model,  generation  of  performance  data  through  the  use  of 
those  applications,  and  feeding  the  results  back  into  the  Micro  SAINT  model 
for  ftirther  refinement. 
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PREFACE 


This  report  provides  a  summaiy  and  overview  of  the  Risk.  Assessment  Tools 
contract.  The  contract,  which  ran  from  July  1990  to  March  1994,  contained  a 
variety  of  efforts  by  individuals  in  four  corporations:  Horizons  Technology,  Inc.; 

NTI,  Inc.;  Pacific  ^erra  Research  Corporation,  and  Georgetown  University.  Detail 
on  the  individual  efforts  is  kept  minimal  here,  and  may  be  found  in  the  many 
reports  and  manuals  generated  during  the  contract's  progress. 

The  authors  would  especially  like  to  acknowledge  the  contributions  of  Mo.  Carol 
Oles  and  Mr,  Mark  Gianturco  of  Horizons  Technology,  Dr.  Samuel  (Mcgor)  Moise 
and  Mr.  Joe  Campbell  of  NTI,  Mr.  George  Anno  and  Dr.  Gene  McClellan  of  Pacific 
Sierra  Research,  and  Dr.  Gary  Kay  of  Geoi^setown  University. 

In  particular,  HTI  appreciates  the  support  and  guidance  provided  to  the  contract 
team  by  Dr.  Fred  Hegge  of  the  Office  of  Military  Performance  Assessment 
Technology,  U.  S.  Army  Medical  Research  and  Development  Command,  and  Mr. 
Rob  Kehlet  of  the  Radiation  and  Policy  Division,  Defense  Nuclear  Agency. 
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Actor®  is  a  registered  trademark  of  Slymantec  Corporation 
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SECTION  1 


INTRODUCTION 

This  is  the  Final  Report  for  Contract  No.  DNA001-90-C-0118,  Risk  Assessment 
Tools.  This  document  describes: 

□  overall  contract  objectives  and  contract  tasks; 

□  the  steps  designed  to  accomplish  these  objectives;  and 

□  accomplishments  through  31  March,  1994 

1 .1  CONTRACT  BACKGROUND  AND  CONTEXT. 

The  Risk  Assessment  Tools  contract  was  awarded  to  a  team  headed  by  Horizons 
Technology,  Inc.  (HTI)  by  the  Defense  Nuclear  Agency  (DNA)  on  19  July  1990  for  a 
period  of  39  months.  It  was  subsequently  extended  to  31  March,  1994. 

Funding  for  the  contract  was  provided  jointly  by  the  DNA  and  the  U.S.  Army 
Medical  Research  and  Development  Command  (USAMRDC)  Office  of  Military 
performance  Assessment  Technology  (OMPAT).  HTI  and  its  subcontractors 
{Georgetown  University  (GTU);  NTl,  Inc.  (NTI);  and  Pacific  Sierra  Research 
Corporation  (PSR)}  were  responsible  for  carrying  out  research  and  development 
activities  to  meet  contract  requirements. 

The  Risk  Assessment  Tools  contract  was  conducted  within  the  framework  of 
previous  work  done  in  the  area  of  medical  defense  against  chemical  weapons  by 
OMPAT,  a  USAMRDC  activity,  and  in  the  area  of  human  response  within  nuclear 
environments  by  the  Radiation  and  Policy  Division  (RARP)  of  the  DNA  Figure  1-1 
presents  an  overvievv  of  the  OMPAT  risk  assessment  tools  development  program. 

The  OMPAT  program's  goal  is  to  support  military  users  with  historical  data  and 
reports  including  estimates  of  human  performance  decrements.  This  information  is 
derived  from  a  number  of  research  activities  including: 

U  analysis  of  military  work/tasks,  conditions,  and  requirements; 

□  measurement  of  performance  using  standardized  controlled  tests; 

□  estimates  of  risk  to  military  system  performance; 

□  development  of  performance  norms  and  standards; 

□  verification  of  system  performance  risk  estimates  using  field  exercises, 
simulations,  and  computer  models;  and 

□  archiving  research  reports,  test  results,  and  bibliographies. 
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Figure  1-1.  Program  overview. 


1 .2  CONTRACT  OBJECTIVES  AND  TASKS. 


There  were  two  major  objectives  for  the  Risk  Assessment  Tools  contract: 

(1)  to  consolidate  efforts  by  the  USAMRDC  and  the  DNA  to  assess  risks  to 
personnel  in  nuclear,  biological,  and  chemical  (NBC)  environments;  and 

(2)  to  place  resulting  products  in  the  hands  of  decision  makers  responsible  for 
operational  issues  on  the  integrated  battlefield. 

To  accomplish  these  meyor  contract  objectives,  HTI  and  its  subcontractors 
performed  the  following  technical  tasks: 

Task  #1.  Incorporate  Human  Response  Performance  (HRP)  data  into  the 
OMPAT  Performance  Infonnation  Management  System  (PIMS); 

Task  #2.  Expand  the  structured  risk  assessment  database,  i.e.,  the 
Taxonomic  Work  Station; 

Task  #3.  Develop  a  Synthetic  Task  Authoring  System  (SYNTAS)  to 

create  part-task  simulators; 

Task  M.  Identify  and  exercise  models  as  a  "test  bed"  for  risk  assessment 
tools; 

Task  #5.  Integrate  PC-based  risk  assessment  tools;  and 
Task  #6.  Conduct  conferences  and  user  workshops. 

1.3  SCHEDULING  OF  TASKS. 

During  FY90/91,  the  focus  was  on  incorporating  HRP  data  into  the  PIMS  (Task  #1), 
evaluating  the  Taxonomic  Workstation  software  (Task  #2),  designing  a  tool  to 
create  part-task  simulators  (Task  #3),  and  identifying  and  exercising  models  as  a 
test  bed  for  the  risk  assessment  tools  (Task  #4).  During  FY92,  the  focus  shifted  to 
collecting  human  response  data  for  incorporation  into  the  PIMS  (Task  #1), 
developing  a  tool  to  create  part-task  simulators  (Task  #3),  and  initiating  steps  to 
integrate  the  PC-based  risk  assessment  tools  (Task  #5).  In  the  final  year  of  the 
contract  -  FY92/93,  the  focus  was  on  incorporating  the  HRP  data  into  the  PIMS 
(Task  #1),  completing  the  tool  for  creating  part-task  simulators  (Task  #3),  and 
integrating  the  tools  (Task  #6).  Task  #6,  conducting  conferences  and  user 
workshops,  was  a  mcgor  emphasis  in  FY92  and  continued  in  FY93  as  products  were 
delivered. 
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The  progressive  design  of  the  Synthetic  Task  Authoring  System  had  a  mcgor  impact 
on  the  focus  of  the  entire  project.  On  initiation  of  the  contract  the  intent  of  Task  #3 
was  to  revise  and  upgrade  the  software  environment  of  the  Unified  Tri-service 
Cognitive  Performance  Assessment  Battery  (UTG-PAB),  developed  under  a 
preceding  contract.  At  the  contract  kick-oft  meeting  in  September  1990,  the  NTI 
representative,  Dr.  Moise,  suggested  an  alternative  approach,  that  of  developing  an 
extendible,  modular,  object-oriented  authoring  tool  capable  of  producing  software 
modules  providing  the  hinctionaliiy  of  the  UTC-PAB.  This  approach  was 
subsequently  approved  by  OMPAT  and  DNA  and  adopted  by  the  team. 

The  authoring  tool,  subsequently  named  the  Synthetic  Task  Authoring  System 
(SYNTAS)  grew  in  scope  as  its  design  progressed  and  its  potential  was  realized. 
Ultimately  it  became  a  computer-assisted  software  engineering  (CASE)  tool  capable 
of  producing  a  wide  variety  of  assessment  instruments.  In  addition,  its  increasing 
capabilities  covered  several  of  the  components  of  other  tasks  in  the  contract. 
Therefore  the  increased  scope  of  Task  #3  was  balanced  by  scaling  back  the  scope  of 
several  of  the  other  tasks,  specifically  Tasks  #2  and  #5. 

1 .4  ORGANIZATION  OF  THE  REPORT. 

This  report  contains  six  major  sections,  one  for  each  of  the  six  technical  tasks 
identified.  Within  each  task  section,  the  most  relevant  tasks  from  the  statement  of 
work  (SOW)  are  listed. 

The  reader  sh  ould  note  that  the  six  tasks  in  the  revised  structure  are  not  numbered 
or  presented  in  the  same  order  as  the  SOW  tasks;  nonetheless,  all  eight  SOW  tasks 
are  included  in  this  plan  under  at  least  one  of  the  six  revised  tasks.  (For  ease  of 
creating  an  audit  trail  back  to  the  SOW,  the  referenced  SOW  tasks  continue  to  be 
numbered  as  they  were  in  the  SOW.) 

Within  each  task  section,  the  following  information  is  provided; 

□  Relevant  SOW  Tasks:  listing  tasks  from  the  original  SOW  that  are  covered 
by  the  task; 

□  Specific  Objective(s):  a  statement  of  specific  objectives  related  to  the  task; 

□  Steps  to  Accomplish  Objectives:  activities  and  steps  used  to  accomplish  the 
SOW  objectives; 

□  Discussion  and  Task  Evaluation:  a  brief  chronology  of  the  task  evolution  and 
related  issues; 

□  Deliverables:  a  listing  of  research  products  and  supporting  documents 
delivered; 

□  Responsible  Contractors:  those  contractors  carrying  out  activities  to  meet 
task  objectives. 
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SECTION  2 


TASK  #1  INCORPORATE  HRP  DATA  INTO  THE  PIMS 

DATABASE 

2.1  RELEVANT  SOW  TASKS. 

The  following  tasks  in  the  SOW  were  addressed  by  Task  #1. 

SOW  Task  1.  Collect  data  and  develop  performance  norms 
SOW  Task  6.  Develop  and  support  the  PIMS. 

2.2  SPECIFIC  OBJECTIVES. 

The  objectives  of  Task  #1  were  to  collect  and  incorporate  appropriate  Human 
Response  Program  (HRP)  documents  and  data  into  the  PIMS  database,  and  to 
design  a  tailored  hierarchical  structure  for  accessing  and  retrieving  the  PIMS  HRP 
data. 

2.3  STEPS  TO  ACCOMPLISH  OBJECTIVES. 

Step  1.  HTI  representatives  met  with  OMPAT  and  DNA  program  managers  to 
discuss  and  outline  the  structure  of  the  PIMS.  This  overall  structure  is 
illustrated  in  Figure  2-1.  The  focus  of  Task  #1  was  on  the  PIMS  text 
and  associated  numeric  database  information  which  includes  ASCII 
input  from  books,  reports,  and  bibliographies. 

Step  2.  Identify  and  catalogue  HRP-related  documents  and  data. 

a.  Catalogue  HRP-related  documents  in  the  DNA  library. 

b.  Identify  and  catr  :ogue  other  HRP-related  documents  and  data  including: 

□  DNA  contractor  references^data 

□  DASAIC  databases 

□  Other  relevant  databases 

Step  3.  Produce  bibliography  and  database  of  HRP-related  documents. 

a.  Compile  initial  draft  bibliography  and  database  of  documents  from  DNA 
library. 

b.  Integrate  other  references  into  bibliography  and  database. 

c.  Establish  an  ongoing  tracking  system  via  NERAC  to  identify  new  HRP- 
related  documents  as  they  become  available. 
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Overview  of  the  performance  information  management  system 
(PIMS)  structure. 


Stop  4.  Prioritize  HKP  documente  for  entry  into  the  PIMS  database  including: 


□  DNA  library  documents 

□  Other  contractor  reports 

□  DASAIC  references 

□  Other  database  references 

Stop  5  Scan  selected  HRP  documents  and  incorporate  them  into  the  PIMS. 

Stop  6.  Collect  and  evaluate  HRP-related  chemical  and  radiation  data  during 
the  Close  Combat  Light  (CCL)  exercises  at  Fort  Huntor-Liggett, 
California. 

a.  Design,  pilot  teat,  and  revise  questionnaires  for  collecting  HRP-related 
chemical  and  radiation  data. 

b.  Administer  questionnaires  to  IIBRAVO  and  13BRAVO  subject  matter 
experts. 

c.  Perform  snalyses  of  questionnaire  data  and  determine  coefficients  for 
performance  prediction  algorithms. 

d.  Produce  technical  report,  software  data  base,  and  computational  algorithms 
for  predictions  of  expected  combat  performance  levels  if  exposed  to  degrading 
effects  of  nuclear  radiation  or  chemical  agents. 

e.  Integrate  technical  report,  data  base,  and  computational  algorithms  into 
PIMS. 

Stop  7  Organize  the  data  in  the  PIMS  database  to  include  HRP  data  fiorn  ONA 
in  addition  to  that  provided  by  OMPAT. 

Step  8.  [Added  during  the  contract]  Collect  and  evaluate  HRP-related  crew 

time  and  motion  data  on  M198  howitzer  crews  in  MOPP4  during  Marine 
Corps  exercises  at  Aberdeen  Proving  Ground,  Maryland. 

2.4  DISCUSSION  AND  TASK  EVOLUTION. 

At  initiation  of  the  contract,  a  prototype  PIMS  was  in  operation  at  OMPAT.  It 
utilized  the  document  management  product  Textware*'  and  contained  a  substantial 
amount  of  information  on  Chemical  Risk  Assessment.  Initial  focus  was  to  extend 
this  qrstem  to  include  information  from  the  nuclear  risk  assessment  arena. 

At  first,  HTI  planned  to  capture  selected  data  from  DNA,  scan  it  into  the  computer, 
perform  optical  character  recognition  on  the  resultant  image,  then  organize  the 
information  and  incorporate  it  into  Textware. 
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In  preparation  for  this  effort,  HTI  conducted  an  analyeie  of  hardware  and  software 
available  for  the  task  and  recommended  that  DNA-RARP  procure  hardware  (an 
IBM  PC'Compatible,  386-based  PC;  a  F^iit8u  scanner,  a  laser  printer,  an  optical 
disk  subs)rstem)  and  software  (Textware  with  Images,  OmniPage  OCR  sof^are)  to 
support  data  capture  and  integration. 

Early  in  1991,  OMPAT  determined  that  the  Textware  system  was  not  adequate  to 
\xse  as  the  basis  for  the  PIMS,  and  began  converting  the  OMPAT  PIMS  to  another 
product  called  HYPLUS. 

Lengthy  delays  were  experienced  in  the  deliveiy  of  DNA's  scanning  hardware  and 
software,  and,  once  delivered,  the  scanning  hardware  and  software  presented  a 
number  of  problems.  While  awaiting  resolution  of  these  problems,  HTI  developed 
tentative  protocols  for  scanning,  converting,  and  correcting  documents.  In  addition, 
Hn  conducted  a  comprehensive  audit  of  material  in  the  DNA  HRP  library, 
designed  and  implemented  a  database,  and  produced  a  complete  on-line 
bibliography  of  materials. 

Plans  to  use  a  DNA  summer  hire  to  scan/dean-up  HRP  documents  were  cancelled 
due  to  problems  with  the  scanner.  Instead,  in'!  began  scanning  several  key  HRP 
documents  for  indiision  into  PIMS.  In  1993,  DNA  awarded  a  new  DASIAC  contract 
which  included  the  requirement  to  scan  and  archive  portions  of  the  DASIAC 
collection,  which  included  many  of  the  key  documents  in  the  HRP.  In  order  to  avoid 
duplication  of  effort  and  avoid  incompatible  ^sterns,  no  further  work  scanning  and 
archiving  was  done  under  the  Risk  Assessment  Tools  contract,  with  the 
understanding  that  key  HRP  documents  would  be  scanned  and  archived  at  DASIAC 
and  made  available  to  RARP. 

Two  unforeseen  opportunities  arose  for  DNA/OMPAT  to  collect  HRP-related,  one  at 
the  CCL  exerdses  at  Fort  Hunter-Li^(ett  in  California  and  another  at  Aberdeen 
Proving  Ground,  Maryland.  These  data  would  augment  the  DNA/IDP  performance 
data  for  motorized  Army  combat  personnel  previously  identified  for  incorporation 
into  the  PIMS  database.  In  order  to  take  advantage  of  these  opportunities.  Steps  #5 
and  #6  (outlined  above)  were  added  to  the  Task  #1  plan  and  several  new  products 
were  added  to  the  delivery  schedule.  The  costs  for  these  efforts  were  shared  among 
several  contracts  managed  by  RARP. 

Given  these  events,  the  scheduling  and  prioritization  of  Task  #1  activities  was 
revised.  This  resulted  in  a  change  of  focus  from  scanning  and  database 
organization  to  the  collection  of  valuable  HRP-related  data  at  Fort  Hunter-Liggett 
and  Aberdeen  Proving  Ground. 
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SUMMARY 


Risk  assessment  has  traditionally  been  accomplished  through  a  variety  of  means. 
Analysis  of  military  tasks,  environments,  and  requirements  has  provided  insight 
into  the  cognitive  components  of  work  and  has  allowed  the  creation  of  models. 
Models,  particularly  sequential  network  models,  have  then  been  used  to  simulate 
the  work  tasks  under  varying  conditions  and  performance  levels  to  make  estimates 
of  the  military  effect  of  degradation  of  task  components.  Observation  of  soldiers  in 
the  field,  coupled  with  administration  of  questionnaires,  has  provided  estimates  of 
the  potential  degradation  of  performance  for  a  variety  of  factors,  such  as  pain, 
overheating,  and  nausea  commonly  associated  by  battlefield  weapons  of  mass 
destruction.  Measurement  of  performance  using  standardized  controlled  tests  have 
provided  quantitative  data  to  back  up  these  estimates  and  has  provided  verification 
of  the  risk  estimates.  These  tests,  traditionally  assembled  into  more  generic 
performance  assessment  batteries,  have  progressed  in  recent  years  to  computer- 
based  simulations  of  work-related  tasks. 

This  report  describes  efforts  conducted  under  the  Risk  Assessment  Tools  contract 
design  and  develop  a  new  tool,  called  the  Ssmthetic  Task  Authoring  Syotem,  for 
creating  synthetic  tasks  or  part-task  simulators.  It  also  describes  efforts  to 
improve,  automate,  correlate,  and  provide  better  access  to  a  variety  of  tools. 

The  Risk  Assessment  Tools  contract  not  only  continued  the  important  work  done  by 
DNA’s  Intermediate  Dose  Program  but  produced  an  extensible  authoring  tool, 
SYNTAS,  for  test  instruments  that  will  simplify  the  data  gathering  phase  of 
subsequent  work. 

SYNTAS  gives  DNA: 

□  A  practical  way  to  estimate  soldier  performance  decrements  due  to  nuclear, 
chemical,  biological,  or  conventional  stresses 

□  An  extensible  platform  for  developing  models,  gaming  scenarios,  training 
systems,  and  assessment  instruments  with  human  performance  included 

□  An  exportable  product  to  other  agencies  and  services  for  similar  uses. 

SYNTAS  has  been  shown  to  be  capable  of  producing  quite  complex  part-task 
simulators.  However,  to  date  no  organization  has  us^  SYNTAS  for  actual 
construction  of  research  instruments.  This  is  partially  due  to  the  very  closed 
development  and  review  community. 

We  recommend  that  efforts  be  made  to  promote  SYNTAS  outside  the  immediate 
development  community  and  encourage  its  practical  use. 
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The  initial  implementation  of  SYNTAS  centers  on  two-dimensional  simulations  on 
a  computer  screen.  This  is  adequate  for  a  large  range  of  tasks  but  stops  short  of 
SYNTAS’  potential. 

Logical  extensions  to  SYNTAS  can  provide: 

□  A  mecms  of  modeling  crew  and  team  performance  using  multiple 
interconnected  SYNTAS  applications 

□  Increased  fidelity  throxigh  incorporation  of  video,  high-quality  sound,  and 
techniques  from  virtual  reality 

□  Through  a  linkage  to  Micro  SAINT,  authoring  of  SYNTAS  applications  based 
on  Micro  SAINT  model,  generation  of  performance  data  through  the  use  of 
those  applications,  and  feeding  the  results  back  into  the  Micro  SAINT  model 
for  further  refinement. 
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PREFACE 


This  report  provides  a  summary  and  overview  of  the  Risk  Assessment  Tools 
contract.  The  contract,  which  ran  fh>m  July  1990  to  March  1994,  contained  a 
variety  of  efforts  by  individuals  in  four  corporations:  Horizons  Technology,  Inc.; 

NTI,  Inc.;  Pacific  Sierra  Research  Corporation,  and  Georgetown  University.  Detail 
on  the  individual  efforts  is  kept  minimal  here,  and  may  be  found  in  the  many 
reports  and  manuals  generated  during  the  contract’s  progress. 

The  authors  would  especially  like  to  acknowledge  the  contributions  of  Mo.  Carol 
Oles  and  Mr.  Mark  Gianturco  of  Horizons  Technology,  Dr.  Samuel  (Migor)  Moise 
and  Mr.  Joe  Campbell  of  NTI,  Mr.  George  Anno  and  Dr.  Gene  McClellan  of  Pacific 
Sierra  Research,  and  Dr.  Gary  Kay  of  Georgetown  University. 

In  particular,  HTI  appreciates  the  support  and  guidance  provided  to  the  contract 
team  by  Dr.  Fred  Hegge  of  the  Offlra  of  Military  Performance  Assessment 
Technology,  U.  S.  Army  Medical  Research  and  Development  Command,  and  Mr. 
Rob  Kehlet  of  the  Radiation  and  Policy  Division,  Defense  Nuclear  Agency. 
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TextWare™  is  a  trademark  of  theTextware  Corporation 
Actor®  is  a  registered  trademark  of  Symantec  Corporation 
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SECTION  1 


INTRODUCTION 

This  is  the  Final  Report  for  Contract  No.  DNA001-90-C-0118,  Risk  Assessment 
Tools.  This  document  describes; 

□  overall  contract  objectives  and  contract  tasks; 

□  the  steps  designed  to  accomplish  these  objectives;  and 

□  accomplishments  through  31  March,  1994 

1 .1  CONTRACT  BACKGROUND  AND  CONTEXT. 

The  Risk  Assessment  Tools  contract  was  awarded  to  a  team  headed  by  Horizons 
Technology,  Inc.  (HTI)  by  the  Defense  Nuclear  Ageni^  (DNA)  on  19  July  1990  for  a 
period  of  39  months.  It  was  subsequently  extended  to  31  March,  1994. 

Funding  for  the  contract  was  provided  jointly  by  the  DNA  and  the  U.S.  Army 
Medical  Research  and  Development  Command  (USAMRDC)  Office  of  Milita^ 
performance  Assessment  Technology  (OMPAT).  HTI  and  its  subcontractors 
{Georgetown  University  (GTU);  NTl,  Inc.  (NTI);  and  Pacific  Sierra  Research 
Corporation  (PSR)}  were  responsible  for  carrying  out  research  and  development 
activities  to  meet  contract  requirements. 

The  Risk  Assessment  Tools  contract  was  conducted  within  the  framework  of 
previous  work  done  in  the  area  of  medical  defense  against  chemical  weapons  by 
OMPAT,  a  USAMRDC  activity,  and  in  the  area  of  human  response  within  nuclear 
environments  by  the  Radiation  and  Policy  Division  (RARP)  of  the  DNA.  Figure  1-1 
presents  an  overvien^  of  the  OMPAT  risk  assessment  tools  development  program. 

The  OMPAT  program's  goal  is  to  support  military  users  with  historical  data  and 
reports  including  estimates  of  humnn  performance  decrements.  This  information  is 
derived  from  a  number  of  research  activities  including: 

□  analysis  of  military  work/tasks,  conditions,  and  requirements; 

□  measurement  of  performance  using  standardized  controlled  tests; 

□  estimates  of  risk  to  military  system  performance; 

□  development  of  performance  norms  and  standards; 

□  verification  of  system  performance  risk  estimates  using  field  exercises, 
simulations,  and  computer  models;  and 

□  archiving  research  reports,  test  results,  and  bibliographies. 
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.  Program  overview. 


1 .2  CONTRACT  OBJECTIVES  AND  TASKS. 


There  were  two  mcgor  objectives  for  the  Risk  Assessment  Tools  contract: 

(1)  to  consolidate  efforts  by  the  USAMRDC  and  the  DNA  to  assess  risks  to 
personnel  in  nuclear,  biological,  and  chemical  (NBC)  environments;  and 

(2)  to  place  resulting  products  in  the  hands  of  decision  makers  responsible  for 
operational  issues  on  the  integrated  battlefield. 

To  accomplish  these  msgor  contract  objectives,  HTI  and  its  subcontractors 
performed  the  following  technical  tasks: 

Task  #1.  Incorporate  Human  Response  Performance  (HRP)  data  into  the 
OMPAT  Performance  Information  Management  System  (PIMS); 

Task  #2.  Expand  the  structured  risk  assessment  database,  i.e.,  the 
Taxonomic  Work  Station; 

Task  #3.  Develop  a  Synthetic  Task  Authoring  System  (SYNTAS)to 
create  part-task  simulators; 

Task  #4.  Identify  and  exercise  models  as  a  "test  bed”  for  risk  assessment 
tools; 

Task  #5.  Integrate  PC-based  risk  assessment  tools;  and 
Task  #6.  Conduct  conferences  and  user  workshops. 

1 .3  SCHEDULING  OF  TASKS. 

During  FY90/91,  the  focus  was  on  incorporating  HRP  data  into  the  PIMS  (Task  #1), 
evaluating  the  Taxonomic  Workstation  software  (Task  #2),  designing  a  tool  to 
create  part-task  simulators  (Task  #3),  and  identifying  and  exercising  models  as  a 
test  bed  for  the  risk  assessment  tools  (Task  #4).  During  FY92,  the  focus  shifted  to 
collecting  human  response  data  for  incorporation  into  the  PIMS  (Task  #1), 
developing  a  tool  to  create  part-task  simulators  (Task  #3),  and  initiating  steps  to 
integrate  the  PC-based  risk  assessment  tools  (Task  #5).  In  the  final  year  of  the 
contract  -  FY92/93,  the  focus  was  on  incorporating  the  HRP  data  into  the  PIMS 
(Task  #1),  completing  the  tool  for  creating  part-task  simulators  (Task  #3),  and 
integrating  the  tools  (Task  #5).  Task  #6,  conducting  conferences  and  user 
workshops,  was  a  megor  emphasis  in  FY92  and  continued  in  FY93  as  products  were 
delivered. 
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The  progressive  design  of  the  Synthetic  Task  Authoring  System  had  a  migor  impact 
on  the  focus  of  the  entire  project.  On  initiation  of  the  contract  the  intent  of  Task  #3 
was  to  revise  and  upgrade  the  software  environment  of  the  Unified  Tri-service 
Cognitive  Performance  Assessment  Battery  (UTC~PAB),  developed  under  a 
preceding  contract.  At  the  contract  kick-off  meeting  in  September  1990,  the  NTI 
representative,  Dr.  Moise,  suggested  an  alternative  approach,  that  of  developing  an 
ertendible,  modular,  object-oriented  authoring  tool  capable  of  producing  software 
modules  providing  the  ^nctionalify  of  the  UTC-PAB.  This  approach  was 
subsequently  approved  by  OMPAT  and  DNA  and  adopted  by  the  team. 

The  authoring  tool,  subsequently  named  the  Synthetic  Task  Authoring  System 
(SYNTAS)  grew  in  scope  as  its  design  progressed  and  its  potential  was  realized. 
Ultimately  it  became  a  computer-assisted  software  engineering  (CASE)  tool  capable 
of  producing  a  wide  variety  of  assessment  instruments.  In  addition,  its  increasing 
capabilities  covered  several  of  the  components  of  other  tasks  in  the  contract. 
Therefore  the  increased  scope  of  Task  #3  was  balanced  by  scaling  back  the  scope  of 
several  of  the  other  tasks,  specifically  Tasks  #2  and  #5. 

1 .4  ORGANIZATION  OF  THE  REPORT. 

This  report  contains  six  major  sections,  one  for  each  of  the  six  technical  tasks 
identified.  Within  each  task  section,  the  most  relevant  tasks  from  the  statement  of 
work  (SOW)  are  listed. 

The  reader  should  note  that  the  six  tasks  in  the  revised  structure  are  not  numbered 
or  presented  in  the  same  o  rder  as  the  SOW  tasks;  nonetheless,  all  eight  SOW  tasks 
are  included  in  this  plan  under  at  least  one  of  the  six  revised  tasks.  (For  ease  of 
creating  an  audit  trail  back  to  the  SOW,  the  referenced  SOW  tasks  continue  to  be 
numbered  as  they  were  in  the  SOW.) 

Within  each  task  section,  the  following  information  is  provided: 

□  Relevant  SOW  Tasks:  listing  tasks  from  the  original  SOW  that  are  covered 
by  the  task; 

□  Specific  Objective(s):  a  statement  of  specific  objectives  related  to  the  task; 

C  Steps  to  Accomplish  Objectives:  activities  and  steps  used  to  accomplish  the 
SOW  objectives; 

□  Discussion  and  Task  Evaluation:  a  brief  chronology  of  the  task  evolution  and 
related  issues; 

□  Deliverables:  a  listing  of  research  products  and  supporting  documents 
delivered; 

□  Responsible  Contractors:  those  contractors  carrying  out  activities  to  meet 
task  objectives. 
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SECTION  2 


TASK  #1  INCORPORATE  HRP  DATA  INTO  THE  PIMS 

DATABASE 

2.1  RELEVANT  SOW  TASKS. 

The  following  tasks  in  the  SOW  were  addressed  by  Task  #1. 

SOW  Task  1.  Collect  data  and  develop  performance  norms 
SOW  Task  6.  Develop  and  support  the  PIMS. 

2.2  SPECIFIC  OBJECTIVES. 

The  objectives  of  Task  #1  were  to  collect  and  incorporate  appropriate  Human 
Response  Program  (HRP)  documents  and  data  into  the  PIMS  database,  and  to 
design  a  tailored  hierarchical  structure  for  accessing  and  retrieving  the  PIMS  HRP 
data. 

2.3  STEPS  TO  ACCOMPLISH  OBJECTIVES. 

Step  1.  HTI  representatives  met  with  OMPAT  and  DNA  program  managers  to 
discuss  and  outline  the  structure  of  the  PIMS.  This  overall  structure  is 
illustrated  in  Figure  2-1.  The  focus  of  Task  #1  was  on  the  PIMS  text 
and  associated  numeric  database  information  which  includes  ASCII 
input  from  books,  reports,  and  bibliographies. 

Step  2.  Identify  cuid  catalogue  HRP-related  documents  and  data. 

a.  Catalogue  HRP-related  documents  in  the  DNA  library. 

b.  Identify  and  catr  .ogue  other  HRP-related  documents  and  data  including; 

□  DNA  contractor  references/data 

□  DASAIC  databases 

□  Other  relevant  databases 

Step  3.  Produce  bibliography  and  database  of  HRP-related  documents. 

a.  Compile  initial  draft  bibliography  and  database  of  documents  from  DNA 
library. 

b.  Integrate  other  references  into  bibliography  and  database. 

c.  Establish  an  ongoing  tracking  system  via  NERAC  to  identify  new  HRP- 
related  documents  as  they  become  available. 


Overview  of  the  performance  information  management  system 
(PIMS)  structure. 


Step  4.  Prioritize  HRP  documents  for  entiy  into  the  PIMS  database  including: 


□  DNA  libraiy  documents 

□  Other  contractor  reports 

□  DASAIC  references 

□  Other  database  references 

Step  5  Scan  selected  HRP  documents  and  incorporate  them  into  the  PIMS. 

Step  6.  Collect  and  evaluate  HRP*related  chemical  and  radiation  data  during 
the  Close  Combat  Light  (CCL)  exercises  at  Fort  Hunter-Liggett, 
California. 

a.  Design,  pilot  test,  and  revise  questionnaires  for  collecting  HRP-related 
chemical  and  radiation  data. 

b.  Administer  questionnaires  to  IIBRAVO  and  13BRAVO  subject  matter 
experts. 

c.  Perform  enalyses  of  questionnaire  data  and  determine  coefficients  for 
performance  prediction  algorithms. 

d.  Produce  technical  report,  software  data  base,  and  computational  algorithms 
for  predictions  of  expected  combat  performance  levels  if  exposed  to  degrading 
effects  of  nuclear  radiation  or  chemical  agents. 

e.  Integrate  technical  report,  data  base,  and  computational  algorithms  into 
PIMS. 

Step  7  Organize  the  data  in  the  PIMS  database  to  include  HRP  data  finm  DNA 
in  addition  to  that  provided  by  OMPAT. 

Step  8.  [Added  during  the  contract]  Collect  and  evaluate  HRP-related  crew 

time  and  motion  data  on  M198  howitzer  crews  in  MOPP4  during  Marine 
Corps  exercises  at  Aberdeen  Proving  Ground,  Maryland. 

2.4  DISCUSSION  AND  TASK  EVOLUTION. 

At  initiation  of  the  contract,  a  prototype  PIMS  was  in  operation  at  OMPAT.  It 
utilized  the  document  management  product  "Textware*'  and  contained  a  substantial 
amount  of  information  on  Chemical  Risk  Assessment.  Initial  focus  was  to  extend 
this  system  to  include  information  from  the  nuclear  risk  assessment  arena. 

At  first,  HTI  planned  to  capture  selected  data  from  DNA,  scan  it  into  the  computer, 
perform  optical  character  recognition  on  the  resultant  image,  then  organize  the 
information  and  incorporate  it  into  Textware. 
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In  preparation  for  this  effort,  HTI  conducted  an  analysis  of  hardware  and  software 
available  for  the  task  and  recommended  that  DNA-RARP  procure  hardware  (an 
IBM  PC-oompatible,  386-based  PC;  a  Ftgitsu  scanner,  a  laser  printer,  an  optical 
disk  subsystem)  and  software  (Textware  with  Images,  OmniPage  OCR  software)  to 
support  data  capture  and  integration. 

Early  in  1991,  OMPAT  determined  that  the  TextwaTre  ^stem  was  not  adequate  to 
use  as  the  basis  for  the  PIMS,  and  began  converting  the  OMPAT  PIMS  to  another 
product  called  HYPLUS. 

Lengthy  delays  were  experienced  in  the  delivery  of  DNA's  scanning  hardware  and 
software,  and,  once  delivered,  the  scanning  hardware  and  software  presented  a 
number  of  problems.  While  awaiting  resolution  of  these  problems,  HTI  developed 
tentative  protocols  for  scanning,  converting,  and  correcting  documents.  In  addition, 
HTI  conducted  a  comprehensive  audit  of  material  in  the  DNA  HRP  library, 
designed  and  implemented  a  database,  and  produced  a  complete  on-line 
bibliography  of  materials. 

Plans  to  use  a  DNA  summer  hire  to  scan/dean-up  HRP  documents  were  cancelled 
due  to  problems  with  the  scanner.  Instead,  HTI  began  scanning  several  key  HRP 
documents  for  inclusion  into  PIMS.  In  1993,  DNA  awarded  a  new  DASIAC  contract 
which  included  the  requirement  to  scan  and  archive  portions  of  the  DASIAC 
collection,  which  included  many  of  the  key  documents  in  the  HRP.  In  order  to  avoid 
duplication  of  effort  and  avoid  incompatible  systems,  no  further  work  scanning  and 
archiving  was  done  under  the  Risk  Assessment  Tools  contract,  with  the 
understanding  that  key  HRP  documents  wotild  be  scanned  and  archived  at  DASIAC 
and  made  available  to  RARP. 

Two  unforeseen  opportunities  arose  for  DNA/OMPAT  to  collect  HRP-related,  one  at 
the  CCL  exercises  at  Fort  Hunter-Liggett  in  California  and  another  at  Aberdeen 
Proving  Ground,  Maryland.  These  data  would  augment  the  DNA/IDP  performance 
data  for  motorized  Army  combat  personnel  previously  identified  for  incorporation 
into  the  PIMS  database.  In  order  to  take  advantage  of  these  opportunities.  Steps  #5 
and  #6  (outlined  above)  were  added  to  the  Task  #1  plan  and  several  new  products 
were  added  to  the  delivery  schedule.  The  costs  for  these  efforts  were  shared  among 
several  contracts  managed  by  RARP. 

Given  these  events,  the  scheduling  and  prioritization  of  Task  #1  activities  was 
revised.  This  resulted  in  a  change  of  focus  from  scanning  and  database 
organization  to  the  collection  of  valuable  HRP-related  data  at  Fort  Hunter-Liggett 
and  Aberdeen  Proving  Ground. 
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While  the  exorcises  provided  an  unparalleled  opportunity  to  augment  the  DNA 
Intermediate  Dose  Performance  (IDP)  database,  specific  data  collection  activities 
were  not  originally  budgeted.  Therefore,  it  was  decided  that  Steps  #5  and  #7  - 
scanning  documents  and  organising  the  HRP  data  within  the  PIMS  database  •  be 
accomplished  outside  of  the  Risk  Assessment  Tools  contract.  Such  activities  were 
not  time-critical  and  could  be  carried  out  by  DNA  and  OMPAT  support  staff  and  at 
DASIAC  as  time  and  resources  allowed. 

2.5  DELIVERABLES. 

Documents  and  products  delivered  under  Task  #1  were: 

Ljuting  and  PrioritiKAtion  of  DNA  HRP  Doeumente. 
mi-TR419.004.  Februaiy  1991. 

IDP  performance  data  representing  regression  parameter  and  data  fit 
statistics  for  individual  crew  members  of  the  four  ground  combat  crews  (files 
related  to  DNA-TR-85-52  and  DNA-TR-88-173).  February  1991.  (These  data 
were  developed  by  PSR  under  previous  contracts  and  provided  for  use  in  the 
PIMS  database.) 

dRflaa  SnAwsra  Listing  r.ntAgnrimTig  nf  m>JA  HRP  PociimontM.  HTI-SW- 

419-001.  March  1991. 

Example  Organization  and  Structure  of  Selected  DNA  HRP  Documents 
Using  the  PIMS  HYPLUS  Hypertext  Software.  March  1991. 

NERAC  and  DASAIC  Literature  Searches  Related  to  the  Effects  of  Radiation 
on  Human  Performance  and  the  Development  of  Synthetic  Work  Tasks. 
March  1991. 

Software  for  PSR  PEAB  Version  1.0.  May  1991.  (This  software  was 
developed  by  PSR  under  a  previous  DNA  contract  and  provided  for  review  by 
the  Risk  Assessment  Tools  contract  staff.) 

The  AQUAFHL  Program  QuestionnaiTes  and  Raw  Data.  June  1992. 

AQUAFHL  Task  Perfnmnaiiefl  Regraairiftn  ReletioiiKhipH  September  1992. 

AQUAFHL  Performence  Data  (3D  Dose  X  Time').  12  November  1992;  updated 
22  January  1993. 

Task  Time  Data  Collection  for  M198  Howitzer.  November  1993.  (Funded 
under  a  separate  DNA  contract  to  PSR  and  provided  for  use  in  the  PIMS 
database.) 

AQUAFHL  technical  report,  software  data  base,  and  computational 
algorithms  for  predictions  of  expected  combat  performance  levels  under 
degrading  effects  of  nuclear  radiation  or  chemical  agents. 
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The  Effect  of  MOPP4  on  M198  Howitzer  Crew  Performance,  Volume  1  - 
Emplacement  and  Displacement  Times  and  Bates  of  Fire,  1  November  1993. 

2.6  RESPONSIBLE  CONTRACTORS. 

HTI  provided  management  support,  PIMS  technical  support,  database  design  and 
data  collection.  PSR  designed  and  administered  questionnaires  in  field  settings 
and  generated  reports. 
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SECTION  3 


TASK  #2  -  EXPAND  THE  STRUCTURED  RISK  ASSESSMENT 

DATABASE 


3.1  RELEVANT  SOW  TASKS. 

SOW  Task  2.  Increase  risk  identification  capabilities 

3.2  SPEanC  OBJECTIVE. 

The  ofagective  of  Task  #2  was  to  extend  the  existing  Taxonomic  Work  Station  (TWS) 
database  to  include  HRP>related  data  and  updated  taxonomies. 

3.3  STEPS  TO  ACCOMPLISH  OBJECTIVES. 

Step  1.  Update  existing  TWS  Version  1.0  taxonomies  and  data  including: 

□  Blueprint  of  the  Battlefield 

□  Human  Factors  Parameters 

□  Conditions  Taxonomy 

□  Mission/Task  Inventory 

Step  2.  Input  SAIC  HRP  database  into  the  TWS  Version  1.0. 

Step  3.  Exercise  the  existing  TWS  software  to  gain  familiarity  and  to  evaluate 
its  status. 

□  Determine  for  which  versions  of  Advanced  Revelation  the  existing  TWS  will 
and  will  not  work. 

□  Determine  the  level  of  effort  required  to  upgrade  the  revised  TWS  Version 
1.0  to  the  U.S.  Army  Research  Institute  (ARI)  TWS  Vernon  1.1  and  to  future 
TWS  Versions  2,0  or  higher. 

3.4  DISCUSSION  AND  TASK  EVOLUTION. 

The  TWS  effort  was  originally  scheduled  to  start  in  late  FY91  and  continue 
throughout  FY92;  instead,  Task  #2  efforts  were  postponed.  This  revised  plan 
allowed  HTI  and  its  subcontractors  to  function  within  budget  constraints,  to 
accommodate  delays  in  receiving  the  Advanced  Revelation  software,  and  to  respond 
to  other  project  task  priorities. 
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Since  the  current  TWS  software  runs  under  Advanced  Revelation  Version  1.0,  a 
decision  was  made  at  the  IPR  #2  to  initially  update  existing  TWS  taxonomies  and  to 
input  the  SAIC  HRP  database  (Steps  1  and  2)  using  TWS  Version  1.0. 

Prior  to  the  present  contract  award,  ARI  began  to  revise  the  TWS  to  upgrade  it  to 
Version  1.1  using  Advanced  Revelation  Version  1.1.  Because  of  this  change  in 
existing  software,  ARI  had  already  addressed  the  concerns  of  Task  #2,  Steps  1-3. 
ARI  also  had  recently  developed  an  automated  Blueprint  of  tVie  Battlefield  (updated 
AUTOBOB  or  BOBCAT,  Version  2.0)  which  was  released  during  FY92.  Both  of 
these  actions  affected  the  Risk  Assessment  Tools  contract  priorities  by  eliminating 
the  need  to  implement  the  various  taxonomies  into  the  TWS.  Because  of  the 
availability  of  the  revised  Version  1.1  TWS  and  the  Version  2.0  BOBCAT  for  use  by 
DNA,  OMPAT  and  other  government  clients,  it  was  agreed  at  IPR  #3  that  Task  #2 
activities  be  dropped  from  the  Risk  Assessment  Tools  contract.  Such  a  decision 
would  prevent  duplication  of  effort  across  government  agencies  and  provide 
additional  hinds  for  higher  priority  contract  activities. 

In  place  of  the  revised  TWS,  HTI  agreed  to  acquire  and  deliver  to  DNA  and  OMPAT 
copies  of  TWS  version  1.1  and  the  updated  Automated  Blueprint  of  the  Battlefield. 

3.5  DELIVERABLES. 

Products  and  supporting  documents  delivered  under  Task  #2  included 

Taxonomic  Work  Station  Version  1.1,  March  1993  (Provided  by  the  Army 
Research  Institute) 

Blueprint  of  the  Battlefield,  Version  2.0  (Provided  by  the  Army  Research 
Institute) 

3.6  RESPONSIBLE  CONTRACTORS. 

This  task  was  accomplished  by  HTI,  with  most  input  from  outside  the  contract  by 
ARI. 
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SECTION  4 


TASK  #3  -  DEVELOP  A  TOOL  TO  CREATE 
PART-SIMULATORS 

4.1  RELEVANT  SOW  TASKS. 

SOW  Task  5.  Revise  UTC-PAB  software  to  include  creation  of  utilities  and 

enhancements. 

4.2  SPECIFIC  OBJECTIVES. 

The  objective  of  Task  #3  was  to  design  and  develop  a  software  system  (referred  to  as 

the  Synthetic  Task  Authoring  System  or  SYNTAS)  that  can: 

□  create,  develop,  and  implement  part-task  simulators  and  synthetic  work 
tasks  for  measuring  human  performance  on  military  tasks; 

□  reflect  sequential  network  modeling  concepts  and  the  nodal  structure  and 
interrelationships  of  military  tasks;  and 

□  be  capable,  modular,  extensible,  and  maintainable. 

4.3  STEPS  TO  ACCOMPLISH  OBJECTIVES. 

Step  1.  Develop  proposal  for  updating  the  UTC-PAB  to  meet  the  objectives 
stated  above. 

Step  2.  Develop  SYNTAS  functional  specifications  and  demonstration  user 

interface.  This  included  functional  design  of  the  software,  evaluation  of 
software  tools,  hardware  and  software  requirements,  cost  estimates,  and 
a  proposed  development  schedule. 

Step  3.  Develop  SYNTAS  system  specification  and  concept  demonstration  to 
provide  a  detailed  definition  of  SYNTAS  functionality  and  an 
illustration  of  how  a  synthetic  task  can  be  implemented.  This  would 
serve  as  a  basis  for  system  testing  and  acceptance. 

Step  4.  Develop  SYNTAS  software  specifications. 

Step  5.  Program,  test  and  document  SYNTAS  software  according  to  software 
speciftcations. 

a.  Develop  draft  SYNTAS  user  interface  and  designer's  guide. 

b.  Define  criterion  test. 

c.  Develop  and  test  SYNTAS  Alpha  Version  1. 
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□  Alpha  Authoring  System  software  and  designer's  guide. 

□  Alpha  Real-Time  System  software  and  User's  Guide. 

□  Alpha  Integrated  Authoring  and  Runtime  System  software  and 
designer's  guide. 

□  Alpha  Integrated  Authoring,  Runtime,  and  Real-Time  System 
software  and  Designer's  Guide. 

□  Interact  with  and  respond  to  inquiries/feedback  from  the  Alpha 
Test  Group,  releasing  periodic  updates  to  the  software  and 
documentation. 

d.  Develop  and  test  SYNTAS  Beta  Version  1 

□  Integrated  SYNTAS  Authoring  and  Runtime  System  software  and 
designer's  guide. 

□  Integrated  SYNTAS  Authoring,  Runtime,  and  Real-Time  System 
software  and  designer's  guide. 

□  Interact  with  and  respond  to  inquiries/feedback  from  the  Beta 
Test  Group,  releasing  periodic  updates  to  the  software  and 
documentation. 

f.  Develop  final  SYNTAS  Version  1  software,  technical  software  documentation, 
and  user  guide. 

4.4  DISCUSSION  AND  TASK  EVOLUTION. 

Initially,  this  task  centered  on  updating  and  revising  the  existing  UTC-PAB 
software.  At  the  initial  project  planning  meeting  held  at  HTI  in  September  1990, 
NTl  recommended  changing  this  task  to  produce  an  authoring  system  capable  of 
creating  the  components  of  the  UTC-PAB  while  addressing  a  number  of 
compatibility  and  performance  issues.  This  approach  was  subsequently  approved 
byDNAandOMPAT. 

During  the  first  year  of  the  contract,  NTI  conducted  a  detailed  study  of  technical 
approaches,  evaluated  computer-assisted  software  tools  and  candidate 
programming  languages,  and  recommended  to  the  planning  group  that  an  object- 
oriented  authoring  system  based  on  a  graphical  user  interface  be  implemented 
under  Microsoft  Windows  using  the  Actor  programming  language.  Subsequently, 
NTI  developed  a  functional  specification  and  softwcure  specification  for  SYNTAS. 

Development  of  SYNTAS  software  began  in  the  second  year  of  the  contract. 
Limitations  in  the  Windows  task  scheduling  system  (specifically  its  inability  to 
perform  deterministic,  preemptive  scheduling  of  tasks)  coupled  with  the  need,  for 
millisecond  accura^  in  the  test  instruments  produced  caused  a  splitting  of  the 
software  development  into  two  parts  -  an  Authoring  System  implemented  in  Actor 
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to  execute  under  Microsoft  Windows,  and  a  Real-time  System  implemented  in  C 
and  executed  under  MS-DOS.  The  development  of  the  Authoring  System  was 
begun  by  HTl  while  the  development  of  the  Real-Time  system  was  begun  by  NTI. 

As  development  proceeded,  design  of  the  Real-time  system  proved  to  be  quite 
difficult,  stretching  the  capabilities  of  the  PC  and  MS-DOS.  At  a  technical  review 
meeting  held  in  San  Antonio  in  Februaiy  1992,  it  was  decided  that  the  user 
interface,  the  interface  with  the  Authoring  System,  and  other  non-Real-time 
component  such  as  file  I/O  would  be  developed  by  HTI;  leaving  NTI  to  focus  on  the 
real  time  scheduling  component,  subsequently  called  the  Real-Time  Kernel.  HTFs 
component  was  initially  called  the  Utility  code. 

As  HTI  developed  this  utility  component,  it  became  necessaiy  to  test  it.  The  HTl 
programmer  produced  a  non-real-iime  version  of  the  system  to  use  in  testing,  which 
eventually  became  a  product  in  its  own  right,  called  the  Runtime  System.  The 
Runtime  system  combined  the  Utility  component  with  a  simplified,  non-real-time 
scheduler.  When  the  real-time  kernel  was  completed,  it  was  also  combined  with  the 
Utility  compcncnt  to  produce  the  Real-time  System. 

Due  to  the  com  monalty  of  the  Utility  component,  models  produced  by  the  Authoring 
System  would  run  on  either  the  Runtime  or  Real-Time  system  without  modification. 
The  Runtime  System  offered  some  advantages:  it  would  run  under  MS  Windows 
and  thus  could  be  used  to  test  Authoring  System  output  without  leaving  Windows, 
and  it  could  handle  large,  slow  operations  such  as  display  of  large  bitmap  files 
without  causing  the  frame  overruns  that  would  stop  the  Real-Time  System.  Its 
disadvantage  was,  of  course,  its  non-real-time  nature,  i.  e.  there  was  no  guarantee 
that  models  run  on  the  Runtime  Sysvem  would  adhere  to  real-time  timing  and 
scheduling  standards. 

Similarly,  the  Authoring  System  evolved  throughout  the  course  of  the  contact. 
Initially  it  was  to  be  based  on  a  flow-chart  metaphor,  and  would  take  advantage  of 
a  sample  application  shipped  with  Actor  that  implemented  much  of  the  graphical 
procedures  for  flow  charts.  Flow  charts,  however,  had  two  liabilities.  First, 
programming  with  flov;  charts  is  not  less  difficult  than  programming  itself;  all  it 
does  is  substitute  pictures  for  structured  phrases.  Secondly,  the  task  models  to  be 
implemented  under  SYNTAS  were  not  expressed  as  flow  charts,  but  rather  as 
sequential  network  diagrams,  a  metaphor  similar  mathematically  to  PERT  charts. 
Consequently,  the  overall  design  of  the  Authoring  System  moved  from  flow  charts 
to  sequential  network  modeling. 

The  sequential  network  modol,  while  sufficient  for  expressing  the  components  and 
relationships  among  task  components,  is  not  sufficiently  powerful  to  use  as  a 
general  purpose  programming  paradigm.  Through  the  design  process,  the  HTI 
analysts  and  programmers  added  an  event-driven  engine  to  the  network  model, 
producing  a  new  programming  system  capable  of  expressing  complicated  models 
without  the  need  for  traditional  programming  statements.  SYNTAS  truly  became  a 
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tool  designed  for  the  non-programmer,  allowing  those  in  other  disciplines  such  as 
cognitive  p^chology  to  leverage  their  own  expertise  and  efficiently  implement  part- 
task  simulators  and  test  instruments  without  becoming  programmers. 

As  the  SYNTAS  design  evolved,  a  number  of  issues  related  to  the  integration  of 
other  Risk  Assessment  Tools  were  incorporated  into  the  SYNTAS  design.  For 
example,  sequential  network  modeling  structures  and  concepts  were  designed  as  an 
integral  part  of  SYNTAS.  This  concept  was  originally  scheduled  for  investigation 
under  Task  #4,  Modeling,  and  under  Task  #5,  Integration  of  Risk  Assessment  Tools. 
Issues  related  to  Task  #6,  Technology  Transfer  and  User  Workshops,  were 
incorporated  into  the  Task  #3  effort  as  well.  In  another  example,  a  software 
demonstration  of  the  SYNTAS  Authoring  and  Runtime  System  was  presented  at 
the  Second  Annual  Technology  Transfer  Meeting  in  December  1992.  Also,  the 
SYNTAS  Alpha  and  Beta  versions  were  released  to  the  Alpha  and  Beta  Test  Groups 
for  review  and  comment.  These  groups  covered  a  wide  range  of  users  from  the 
Army,  Air  Force,  NASA,  and  universities.  These  types  of  activities  have  created 
interest  and  acceptance  among  potential  users. 

4.5  DELIVERABLES. 

Documents/products  delivered  to  date  under  Task  #3  follow: 

PreliminaTy^  Proposal  for  Modification  of  UTC-PAR  Software.  HTI-TR-419- 
001.  October  1990. 

Functional  Specifications  for  SYNTAS.  HTI-TB-419-002.  Draft,  November 
1990;  Final,  January  1991. 

Demonstration  Software  of  SYNTAS  User  Interface.  January  1991. 

Draft  SYNTAS  System  Specification  and  Concept  Demonstration. 
HTI-TR-419-005.  March  1991. 

Draft  SYNTAS  System  Software  Specifications.  August  1991. 

Demonstration  SYNTAS  Configuration  System  and  User  Interface  software. 
30  January  1992. 

Demonstration  SYNTAS  Real-Time  System  software.  30  January  1992. 

Draft  SYNTAS  (Version  0.6)  User  Interface  software.  16  April  1992. 

Draft  SYNTAS  fVersion  0.5)  Designer’s  Guide.  HTI.419-SW-GUIDE1.  16 
April  1992. 

Draft  SYNTAS  (Version  0.55)  User  Interface  software.  28  May  1992. 

Draft  SYNTAS  fVersion  0.551  Designer's  Guide.  HTI-419-SW-GUIDE/0.65. 
28  May  1992. 
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Modified  demonstration  model  and  subtask  software  for  SYNTAS  Version 
0.55.  8  June  1992. 

Draft  User's  Guide,  SYNTAS  Real-Time  System,  Alpha  VftTaioTi  1.0,  HTI-419- 
SW-RT/GUIDE/1.0.  December  1992. 

Alpha  SYNTAS  Integrated  Authoring  and  Runtime  System  Version  1.0 
software  and  User  Guide.  HTI-419-SW-GUIDE/1.0. 14  January  1993. 

Training  materials  and  conduct  of  SYNTAS  Alpha  Test  Group  Training.  14- 
15  January  1993. 

Alpha  SYNTAS  Integrated  Authoring  and  Runtime  System,  4  February  1992. 

Beta  SYNTAS  Integrated  software  and  Designer's  Guide,  June  1993 

SYNTAS  software.  Version  1.0,  technical  documentation,  and  designer's 
guide,  August  1993 

RTCONFIG  software  and  users  manual,  August  1993 
SCRAMBLE  software  and  users  manual,  August  1993 
EXPAND  software  and  users  manual,  August  1993 

4.6  RESPONSIBLE  CONTRACTORS. 

The  responsible  contractors  for  the  development  of  SYNTAS  were  HTI  and  NTI. 

HTI  provided  overall  program  management;  designed,  coded,  tested,  and 
documented  the  SYNTAS  Authoring  System,  the  Runtime  System,  and  that  portion 
of  the  Real-Time  System  which  is  common  to  both  Runtime  and  Real-Time  Systems 
(e.g.,  user  interface,  graphics,  input/output).  NTI  provided  initial  design  of  the 
system;  authored  the  functional  specification;  and  designed,  coded,  and  tested  the 
Real-Time  Kernel  used  in  the  Real-Time  system.  GTU  raviewed  the  draft 
functional  specification  and  software  specification  and  served  as  SYh^TAS  alpha 
and  beta  testers. 


SECTION  5 


TASK  #4  -  IDENTIFY  AND  EXERCISE  MODELS  AS  A  TEST 
BED  FOR  THE  RISK  ASSESSMENT  TOOLS 

5.1  RELEVANT  SOW  TASKS. 

SOW  Task  3.  Expand  risk  quantification  capabilities 

5.2  SPECIFIC  OBJECTIVES. 

The  objective  of  Task  #4  was  to  identify,  refine,  and  exercise  models  as  a  "test  bed" 
for  the  emerging  risk  assessment  technologies. 

5.3  STEPS  TO  ACCOMPLISH  OBJECTIVES. 

Step  1.  Identify  and  evaluate  existing  task  sequencing  models  and  data  for  use 

as  a  "test  bed"  for  risk  assessment  tools.  This  includes  a  rev  iew  and 
analysis  of  models  and  data  related  to  the  following  ^sterns: 

□  Ml  Abrams  Tank 

□  M60A3Tank 

0  UH-60  Blackhawk  Helicopter 

□  AH*1  Cobra  Helicopter 

□  AH-64  Apache  Helicopter 

Step  2.  Select  2-3  models  and  scenarios  as  "test  bed"  for  the  risk  assessment 
tools,  specifying  the  models  and  scenarios  located,  level  of  task 
specification,  mean  task  times  and  variances,  available  performance 
degradation  data,  model  users,  and  implementation  codes. 

5.4  DISCUSSION  AND  TASK  EVOLUTION. 

Task  #4  activities  (i.e.,  Steps  1  and  2)  were  completed  as  planned.  In  addition,  PSR 
participated  in  the  CCL  exercises  at  Fort  Hunter-Liggett  and  MOPP4  exercises  at 
Aberdeen  proving  Ground  to  gather  HBP-related  chemical,  radiation,  and  heat 
stress  data  in  a  field  setting.  The  data  gathered  provided  input  to  the  Task  #4 
effort. 

A  number  of  different  potential  models  emerged  over  the  course  of  the  contract  as  a 
potential  test  bed  for  the  risk  assessment  tools.  Initially,  the  Task  #4  effort  focused 
on  the  M60A3  Tank  as  a  test  bed.  More  recent  CCL  and  Aberdeen  data  collection 
efforts  have  focused  on  infantiymen  and  artillery  men. 
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In  the  near  future,  there  is  a  need  to  re-evaluate  and  confirm  the  previous  selection 
of  a  single  test  bed  for  future  efforts. 

5.5  DELIVERABLES. 

Documents/products  delivered  under  Task  #4  include: 

An  Asseaament  of  Selected  Crew  Models  as  a  Test  Bed  for  Risk  Assessment 
Tools.  HTI-TR-419-004.  June  1991. 

WAA/HARDMAN  III  Micro  SAINT  models  in  Windows  format  (provided  by 
MicroAnalysis  &  Design).  12  November  1992. 

DNA  Task  Specific  Micro  SAINT  models  in  Windows  format  (provided  by 
MicroAnalysis  &  Design).  12  November  1992. 

Micro  SAINT  models  in  DOS  format  (provided  by  MicroAnalysis  &  De  si  ipi). 
January  1993. 

5.6  RESPONSIBLE  CONTRACTORS. 

Responsible  Contractors  for  this  task  were  HTI,  PSR,  and  GTU.  HTI  provided 
overall  program  management.  PSR  identified,  evaluated,  and  documented  existing 
task  models  and  data  as  a  test  bed  for  the  risk  assessment  tools.  GTU  reviewed  and 
provided  feedback  on  the  written  assessment  report  and  test  bed  model  selection. 
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SECTION  6 


TASK  #5  -  INTEGRATE  PC-BASED  RISK 
ASSESSMENT  TOOLS 

6.1  RELEVANT  SOW  TASKS. 

SOW  Task  4.  Link  SYNTAS,  risk  identification,  and  risk  quantification  capabilities 

SOW  Task  7.  Develop  a  PC-based  computational  aide  for  an  integrated  NBC 
environmental  methodology 

6.2  SPECIFIC  OBJECTIVES. 

The  objectives  of  Task  #5  were  to  integrate  the  PC-based  risk  assessment  tools,  i.e., 
SYNTAS,  TWS,  PIMS,  and  Micro  SAINT;  and  to  encourage  different  user 
populations,  e.g.,  testers,  modelers,  and  ^stem  engineers,  to  consider  a  more 
integrated  approach  which  involves  using  the  fall  range  of  risk  assessment  tools. 

6.3  STEPS  TO  ACCOMPLISH  OBJECTIVES. 

Step  1.  Identify  the  needs  of  extern  users  in  addressing  questions  within  an 
NBC  battlefield  environment. 

a.  Work  with  DNA  and  OMPAT  to  identify  potential  users  of  the  risk 
assessment  tools. 

b.  Conduct  user  discussions  to  ascertain  the  types  of  questions  that  they  are 
asking  or  would  like  to  have  answered  concerning  performance  degradation 
of  militazy  personnel  in  NBC  environments. 

Step  2.  Design  approaches  for  interfacing  between  SYNTAS,  TWS,  PIMS,  and 
Micro  SAINT.  Alternative  approaches  might  involve  the  following: 

a.  Exploring  alternative  taxonomic  structures  as  a  means  of  interfacing  the 
tools,  such  as 

□  literature  on  synthetic  work  tasks,  part-task  simulators,  and 
sequential  network  modeling  existing  TWS  taxonomies 

□  activities  being  conducted  by  OMPAT  and  DNA  (e.g.,  results  of 
the  Perez  analysis) 

b.  Creating  new  TWS  task  structures  using  information  from  the  selected 
models,  for  linking  this  information  into  Micro  SAINT,  and  for  later  storing 
output  from  Micro  SAINT. 

c.  Using  Micro  SAINT  output  to  help  develop  specifications  for  the  design  of 
part-task  simulators  within  SYNTAS. 
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d.  Developing  part<task  simulators  using  SYNTAS. 

e.  Mapping  i^thetic  task  data  back  into  the  TWS  and  into  Micro  SAINT. 

Step  3.  Develop  an  aide  for  integrating  PC-based  risk  assessment  tools. 

6.4  DISCUSSION  AND  TASK  EVOLUTION. 

Task  #5  overlapped  considerably  with  other  project  tasks,  (in  particular  Tasks  #1, 
#3,  #4,  and  #6)  and  a  number  of  activities  related  to  these  other  tasks  dealt  with 
interface  issues.  These  activities  included: 

□  collecting  HRP-related  data  in  field  exercises  and  via  questionnaires  for 
inclusion  in  PIMS,  the  primary  OMPAT  interface  tool  (Task  #1); 

□  gathering  key  DNA  HRP-related  documents  for  inclusion  in  the  PIMS  data 
base  (Task  #1); 

□  incorporating  sequential  network  modeling  and  Petri  modeling 
concepts/structures  into  the  SYNTAS  design  structure  (Task  #3); 

□  interacting  with  the  Alpha  Group  to  identify  user  needs/concems  (Task  #3); 

□  developing  materials  and  participating  in  activities  to  promote  the  SYNTAS 
creating  a  better  understanding  of  user  needs  and  expanding  the  potential 
user  base  to  simulation  and  training  specialists  (Task  #6);  and 

□  reviewing  existing  task  sequencing  models  and  data  as  a  test  bed  for  risk 
assessment  tools  (Task  #4); 

□  developing  materials  and  conducting  training  and  presentations  on  the 
design  of  SYNTAS  models  (Tasks  #3  and  #6). 

These  activities  integrated  concepts  finm  task  analysis  and  sequential  network  and 
other  modeling  techniques  with  the  design  of  performance  assessment  instruments. 
Because  of  these  numerous  activities,  it  was  decided  at  IPR  #3  that  the 
requirements  of  this  task  had  been  met  and  that  the  deliverables  would  be  reduced 
to  a  software  Risk  Assessments  Tools  Introduction  and  a  SYNTAS  demonstration, 
which  was  to  be  implemented  in  SYNTAS  itself. 

6.5  DELIVERABLES. 

Documents  and  products  delivered  under  Task  #5  include: 

SYNTAS  DEMO,  Version  1.0,  a  SYNTAS  application  demonstrating  the 
design  of  models  using  SYNTAS,  December  1993 

6.6  RESPONSIBLE  CONTRACTORS. 

Responsible  Contractor  for  this  task  was  HTI. 


21 


SECTION  7 


TASK  #6  -  CONDUCT  TECHNOLOGY  TRANSFER 
CONFERENCES  AND  USER  WORKSHOPS 

7.1  RELEVANT  SOW  TASKS. 

SOW  Task  8.  Coordinate  and  conduct  user  workshops. 

7.2  SPECIFIC  OBJECTIVES. 

The  objectives  of  Task  #6  were  to  communicate  the  value  of  the  risk  assessment 
tools  in  addressing  performance  degradation  questions  related  to  military 
performance  in  an  NBC  emnronment,  to  transfer  OMPAT/DNA  risk  assessment 
technology  to  military  users,  and  to  train  users  in  this  technology. 

7.3  STEPS  TO  ACCOMPLISH  OBJECTIVES. 

Step  1.  Work  with  DNA  and  OMPAT  to  identify  and  determine  the  needs  of 

potential  users  of  the  risk  assessment  tools. 

Step  2.  Set-up,  coordinate,  host,  conduct,  and  document  the  resiilts  of  annual 
DNA/OMPAT  Technology  Transfer  Meetings. 

a.  Assist  DNA  in  planning  and  conducting  the  Second  Annual  DNA  Technology 
Transfer  Meeting. 

b.  Participate  in  Third  Annual  DNA  Technology  Transfer  Meeting 

op  3.  Develop  and  conduct  workshops  to  train  clients  in  the  use  of 
OMPAT/DNA  risk  assessment  technology. 

7.4  DISCUSSION  AND  TASK  EVOLUTION. 

Each  of  the  risk  assessment  tools  individually  may  have  several  different  types  of 
usc/  ^.  For  example,  there  are  at  least  three  audiences  involved  in  performance 
assessment: 

(1)  designers  of  performance  assessment  instruments  and  synthetic  work  tasks; 

(2)  test  administrators;  and 

(3)  test  subjects. 

In  addition,  audiences  include  designers  and  users  of  training  and  simulation  tools. 
Each  of  these  audiences  has  particular  needs  which  must  be  considered  when  - 
designing  user  guides,  meeting  agenda,  content,  format,  etc. 
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As  SYNTAS  was  being  developed,  it  became  evident  that  there  are  a  number  of 
training  issues  that  go  well  beyond  the  scope  of  the  immediate  contract.  Future 
training  needs  include,  but  are  not  limited  to,  the  use  of  sequential  network 
modeling  concepts  in  test  design  and  the  development  of  more  integrated 
approaches  to  performance  measurement. 

7.5  DELIVERABLES. 

Supporting  documents/products  produced  under  Task  UQ  include: 

DNA  2ND  Annual  Technology  Transfer  Meeting.  3-5  December  1991. 

"Joint  OMPAT/DNA  Risk  Assessment  Program."  Presentation  at  DNA  2nd 
Annual  Technology  Transfer  Meeting.  December  1991. 

"Synthetic  Task  Authoring  System  (SYNTAS)."  Presentation  at  DNA  2nd 
Annual  Technology  Transfer  Meeting.  December  1991. 

Demonstrations  of  SYNTAS  Authoring  System  and  resultant  synthetic  tasks 
at  the  2nd  Annual  DNA  Technology  Transfer  Meeting.  December  1992. 

Training  Session  for  SYNTAS  Alpha  Test  Group.  14-15  Jantiary  1993. 

7.6  RESPONSIBLE  CONTRACTORS. 

Responsible  Contractors  for  this  task  included  HTI,  PSR,  NTI,  and  GTU. 
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